Artificial intelligence applications are frequently used without any mechanism for external testing or evaluation. Modern machine learning systems are opaque to outside stakeholders, including researchers, who can only probe the system by providing inputs and measuring outputs. Researchers, users, and regulators alike are thus forced to grapple with using, being impacted by, or regulating algorithms they cannot fully observe. This brief reviews the history of algorithm auditing, describes its current state, and offers best practices for conducting algorithm audits today. We identified nine considerations for algorithm auditing, including legal and ethical risks, factors of discrimination and bias, and conducting audits continuously so as to not capture just one moment in time.

While machine learning applications in healthcare continue to shape patient-care experiences and medical outcomes, discriminatory AI decision-making is concerning. This issue is especially pronounced in a clinical setting, where individuals' well-being and physical safety are on the line, and where medical professionals face life-or-death decisions every day. Until now, the conversation about measuring algorithmic fairness in healthcare has focused on fairness itself--and has not fully taken into account how fairness techniques could impact clinical predictive models, which are often derived from large clinical datasets. This brief seeks to ground this debate in evidence, and suggests the best way forward in developing fairer ML tools for a clinical setting. We studied the trade-offs clinical predictive algorithms face between accuracy and fairness for outcomes like hospital mortality, prolonged stays in the hospital, and 30-day readmissions to the hospital.

As the development and adoption of AI-enabled healthcare continue to accelerate, regulators and researchers are beginning to confront oversight concerns in the clinical evaluation process that could yield negative consequences on patient health if left unchecked. Since 2015, the United States Food and Drug Administration (FDA) has evaluated and granted clearance for over 100 AI-based medical devices using a fairly rudimentary evaluation process that is in dire need of improvement as these evaluations have not been adapted to address the unique concerns surrounding AI. This brief examined this evaluation process and analyzed how devices were evaluated before approval. We analyzed public records for all 130 FDA-approved medical AI devices between January 2015 and December 2020 and observed significant variety and limitations in test-data rigor and what developers considered appropriate clinical evaluation. When we performed an analysis of a well-established diagnostic task (pneumothorax, or collapsed lung) using three sets of training data, the level of error exhibited between white and Black patients increased dramatically.

Geneva – Biases embedded in artificial intelligence systems increasingly used in healthcare risk deepening discrimination against older people, the World Health Organisation warned. AI technologies hold enormous potential for improving care for older people, but they also carry significant risk, the UN health agency said in a policy brief. "Encoding of stereotypes, prejudice, or discrimination in AI technology or their manifestation in its use could undermine… the quality of health care for older people," it said. The brief highlighted how AI systems rely on large, historical datasets with information about people collected, shared, merged and analysed in often opaque ways. The datasets themselves can be faulty or discriminatory, reflecting for instance existing biases in healthcare settings, where ageist practices are widespread.

The World Health Organization released a policy brief this past week aimed at combating age-related bias in health-related artificial intelligence tools. The brief, "Ageism in artificial intelligence for health," proposes a wide range of measures to ensure older people are effectively engaged in the processes, technologies and services affecting them. "The implicit and explicit biases of society, including around age, are often replicated in AI technologies," said Alana Officer, unit head of demographic change and healthy aging at the WHO. "To ensure that AI technologies play a beneficial role, ageism must be identified and eliminated from their design, development, use and evaluation," Officer continued. As the WHO brief notes, AI carries great potential for transforming healthcare, including on a population-wide level.

In a new policy brief, Ageism in artificial intelligence for health, the agency presents legal, non-legal and technical measures that can be used to minimize the risk of exacerbating or introducing ageism through AI. Artificial intelligence is revolutionizing many fields, including public health and medicine for older people. The technology can help predict health risks and events, enable drug development, support the personalization of care management, and much more. If left unchecked, AI technologies may perpetuate existing ageism in society and undermine the quality of health and social care that older people receive. The data used can be unrepresentative of older people or skewed by past ageist stereotypes, prejudice or discrimination.

Artificial intelligence has been in the spotlight for its ability to discriminate and reflect prejudices against groups of people, be it on the grounds of race, religion, or gender. This, of course, is a result of the prejudices held by the people behind the AI - artificial intelligence is susceptible to the prejudices and discriminatory attitudes held by its creators. Ageism - or prejudice and discrimination on the basis of age - is included in the list as the elderly are continuously neglected in the field of AI, thus excluding their experiences and concerns. This was exactly the point of concern in a recent policy brief by the World Health Organization, which warned that ageism, when exhibited by AI, could have serious impacts on the health of the elderly. "Specifically for older people, ageism is associated with a shorter lifespan, poorer physical and mental health and decreased quality of life," WHO says, adding that it "can limit the quality and quantity of health care provided to older people."

The U.S. Intelligence Community faces a moment of reckoning and AI lies at the heart of it. Since 9/11, America's intelligence agencies have become hardwired to fight terrorism. Today's threat landscape, however, is changing dramatically, with a resurgence of great power competition and the rise of cyber threats enabling states and non-state actors to spy, steal, disrupt, destroy, and deceive across vast distances -- all without firing a shot. The Intelligence Community (IC) faces a moment of reckoning. If the IC cannot adopt AI and other emerging technologies successfully, it risks failure.

This policy brief proposes a group of twenty (G20) coordinating committee for the governance of artificial intelligence (CCGAI) to plan and coordinate on a multilateral level the mitigation of AI risks. The G20 is the appropriate regime complex for such a metagovernance mechanism, given the involvement of the largest economies and their highest political representatives. Other regime complexes and international organizations, which also focus on AI governance, tend to either lack such political power or exclude major rivalry countries. However, such inclusive centrality is necessary to counter the fragmentation of the existing cyber regime complex and effectively coordinate the mitigation of AI risks on a global level. Therefore, the G20 CCGAI is proposed to complement and strengthen polycentric governance as well as AI governance networks. CCGAI's metagovernance function is presented as a task intended to institutionalize linkages between the CCGAI and relevant actors with the G20 complex and the broader AI and cyber regime complex.

Artificial intelligence (AI) in the workplace is most prevalent in the automotive and electronic industries and countries with a particular focus on exports, such as Mexico and developing economies in Asia, are identified as the "most exposed" to an AI influx, according to a policy brief by the UN Conference on Trade and Development published this week in conjunction with research from the World Bank. "The share of occupations that could experience significant automation is actually higher in developing countries than in more advanced ones, where many of these jobs have already disappeared, and this concerns about two thirds of all jobs," the policy brief said. The UN report, published Tuesday, advised that though the use of robots could bring new opportunities for developing economies, it would be important to consider taxing the use of automation which should, in theory, prevent inequality.